I SmRsc'dPCTjm 09NO!/2flflfi 



ATTORNEY'S DOCKET NUMBER 

450101-02537 



TRANSMITTAL LETTER TO THE UNITED STATES 
DESIGNATED/ELECTED OFFICE (DO/EO/US) 
CONCERNING A FILING UNDER 35 U.S.C. 371 



U S. APPLICATION NO. (If ki) 



09/700140 



INTERNATIONAL APPLICATION NO. 

PCT/JP00/01511 



INTERNATIONAL FILING DATE 

MARCH 13, 2000 



(EARLIEST) PRIORITY DATE CLAIMED 

MARCH 12, 1999 



title of invention PICTURE PROVIDING APPARATUS AND PROVIDING METHOD 

THEREFOR, PICTURE PROCESSING APPARATUS AND 
PROCESSSING METHOD THEREFOR, AND MEMORY MEDIUM 

applicant(s) for do/eo/us Tetsujiro KONDO and Junichi ISHIBASHI 



Applicants herewith submit to the United States Designated/Elected Office (DO/EO/US) the following items and other information: 

1 . ^ This is a FIRST submission of items concerning a filing under 35 U.S.C. 371 . 

2. □ This is a SECOND or SUBSEQUENT submission of items concerning a filing under 35 U.S.C. 371 . 

3. ^ This is an express request to promptly begin national examination procedures (35 U.S.C. 371(f)). 

; □ The US has been elected by the expiration of 1 9 months from the priority date (PCT Article 3 1). 

'5- ESI A copy of the International Application as filed (35 U.S.C. 371(c)(2)) 

y a. □ is attached hereto (required only if not communicated by the International Bureau). 

O b. ^ has been communicated by the International Bureau. 

%a. c- Q is not required, as the application was filed m the United States Receiving Office (RO/US). 

=li S An English language translation of the International Application as filed (35 U.S.C. 371(c)(2)). 

S Amendments to the claims of the International Application under PCT Article 19 (35 U.S.C. 371(c)(3)) 

ri a- LZI are attached hereto (required only if not communicated by the International Bureau). 

?= = t>- LH nave been communicated by the International Bureau. 

- ~~ c. □ have not been made; however, the time limit for making such amendments has NOT expired. 

W d. ^ have not been made and will not be made. 

~i£ □ A English language translation of the amendments to the claims under PCT Article 19(35U.S.C371 (c)(3)). 

2 □ An oath or declaration of the inventor(s) (35 U.S.C. 371 (c)(4)). 



□ An English language translation of the 
371(c)(5)). 



3 the International Preliminary Examination Report under PCT Article 36 (35 U.S.C. 



Items 11 to 16 below concern document(s) or information included: 

11. H An Information Disclosure Statement under 37 CFR 1 .97 and 1 .9! 

12. □ An assignment document for recording. A separate cover sheet ir 

13. □ A FIRST preliminary amendment. 

□ A SECOND or SUBSEQUENT preliminary amendment. 

14. D A substitute specification. 

15. Q A change of power of attorney and/or address letter. 

16. □ Oth er items or information : 

PCT/RO/101, PCT/IB/30 1,304,308 
PCT/ISA/220,210 

14 Sheets of Drawings, 1 Page Abstract 



npliance with 37 CFR 3.28 and 3.31 is includ 
EXPRESS MAIL 
Mailing Label Number: EL5850333306US 
Date of Deposit: November 9. 2000 



"Express Mail Post Office to Addressee" Service 
under 37 CFR 1 . 1 0 on the date indicated above and is 
addressed to the Assistant Commissioner for Patents 
and Trademarks, Box PCT Washington, DC 20231. 



(Typed or pnnt 



ie of person mailing paper or fee) 



IS 



Form PTO-1 390 (REV 10-2000) pi 



U.S. APPLICATION NO.flf known, see 37 C.F.R. 1.50) 



422 Ree'd PCT7PTO 0 9 NOV 2000 



09/700140 



INTERNATIONAL APPLICATION NO. 

PCT/JP00/01511 



17.K1 The following fees are submitted 
Basic National Fee (37 CFR 1.492(a)(l)-(5): 

Neither international preliminary examination fee (37 CFR 1.482) 
nor international search fee (37CFR 1.445(a)(2) paid to USPTO 
and International Search Report not prepared by the EPO or JPO 



International preliminary examination fee (37 C.F.R. 1.482) not paid to 
USPTO but International Search Report prepared by the EPO or JPO. . 



International preliminary examination fee (37 CFR 1.482) not p 
international search fee (37 CFR 1.445(a)(2)) paid to USPTO. .. 



ATTORNEY'S DOCKET NO. 

450101-02537 



(CALCULATIONS /pto use only 



International preliminary examination fee paid to USPTO (37 CFR 1.482) 

and all claims satisfied provisions of PCT Article 33(l)-(4) 

ENTER APPROPRIATE BASIC FEE AMOUNT = 



($ 860.00 



Surcharge of $130.00 for furnishing the oath or declaration later than □ 20 □ 30 
months from the earliest claimed priority date (37 CFR 1.492(e)). 


(S 




13 Claims /Number Filed / Number Extra /Rate 


($ 




ToKAClaims / 13 -20= 1 0 /X $18.00 


($ 




InJqv.kkmriaims 8 ■ 3= 5 \ $80.00 


($ 400.00 




MULTIPLE DEPENDENT C'LAlMi S> < if applicable) - S270.00 


(S 




U TOTAL OF ABOVE CALCULATIONS = 


($1,260.00 




□ /Applicant claim' mall entirv staius See 37 ( ' ■■ 1.2" Die lees indicated abo\ 
are reduced by l A. 


($ 




G SUBTOTAL = 


($1,260.00 




Progfssing fee of $130.00 for furnishing the English translation later than □ 20 □ 30 
months from the earliest claimed priority date (37 CFR 1 .492(f)). + 


($ 




f f TOTAL NATIONAL FEE ~- 


($1,260.00 




Fee for recording the enclosed assignments (37 CFR 1.21(h)). The assignment must be 
accompanied by an appropriate cover sheet (37 CFR 3.28, 3.31). $40.00 per property + 


(S 
($ 




TOTAL FEES ENCLOSED = 


($1,260.00 






(Amount to be: 
refunded. 


$ 




(charged: 


$ 



a. A check in the amount of $ 1.260.00 to cover the above ft 

b. □ Please charge my Deposit Account No. intheamc 

A duplicate copy of this sheet is enclosed. 

c. ^ The Commissioner is hereby authorized to charge any additional fees which may be required, or credit any overpayment to Deposit 
Account No. 50-0320 . A duplicate copy of this sheet is enclosed. 

NOTE: Where an appropriate time limit under 37 CFR 1.494 or 1.495 has not been met, a petiSon to revive (37 CFR 1.137(a) or (b)) must be filed and granted to restore the 



SEND ALL CORRESPONDENCE TO: 

WILLIAM S. FROMMER 

FROMMER LAWRENCE & HAUG LLP 

745 FIFTH AVENUE 

NEW YORK, NEW YORK 10151 

Dated: November 9. 2000 



S ION ATT FRF / ^ 



Form PTO- 1390 (REV 10-2000) page 2 of 2 




09/700140 
422 Red'd PCT/PTO 0 9 MOV 2000 



i 



DESCRIPTION 



Picture Providing Apparatus and Providing Method therefor, Picture Processing 
Apparatus and Processing Method therefor, and Memory Medium 

Technical Field 

This invention relates to a picture providing apparatus and a providing method 
therefor, a picture processing apparatus and a processing method thereof, and a 
memory medium adapted so that program for carrying out picture processing is 
recorded, and more particularly to a picture providing apparatus and a providing 
method therefor, and a picture processing apparatus and a processing method therefor 
for providing and processing or displaying picture image at remote place corresponding 
to designation, and a picture providing system using such a picture processing 
apparatus. 

Background Art 

Followed by the fact that basis of information communication is put by 
development of network technology, it has become possible to transmit, on the real 
time basis, data of moving picture through communication network. Under such 
infrastructure, utilization of the technology in which video camera is installed or 
provided at remote place to carry out remote control of that video camera through 
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network to view desired moving pictures is being expected in the field of security, 
advertisement, information guide, real estate, remote medical service and the like. 

As a video camera which permits such a remote control, there are utilized 
pan/tilt video cameras adapted to mechanically move view point of the video camera, 
so called pan or tilt it with a predetennined position being as reference to change 
position of image pick-up to carry out zooming to change angle of visual field. 

Since this pan/tilt video cameras have mechanically movable portion, there are 
the problems that, as compared to video cameras only electronically operative, this 
pan/tilt video camera is inferior in reliability and periodical maintenance is required, 
and because it takes a predetennined time in operation, there exists time lag until a 
desired picture image is transmitted. 

Moreover, the pan/tilt video cameras mechanically operative cannot provide 
image of view points or angles of visual field of image pick-up different from each 
other with respect to plural users. 

Further, at pan/tilt video cameras used in the state installed or provided at 
remote place, in order to permit image pick-up, e.g. , over a broad range of substantially 
360 degrees, image pick-up lens using cone-shaped inirror is used. In the image pick- 
up lens using cone-shaped mirror of this kind, distortion takes place from a structural 
point of view. In view of the above, there are provided various technologies for 
eliininating distortion in picture image data when the cone-shaped mirror is used to 
carry out image pick-up. 
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In addition, there are also provided image pick-up systems in which distortion 
of picture image at the position where user desires of picture images imaged by using 
the image pick-up lens using cone-shaped mirror is removed to transmit such picture 
image to user existing at remote place so that user can obtain desired pan or tilt picture 
image even if camera does not pan or tilt. 

The conventionally proposed technology for eliminating distortion of picture 
data only converts (transforms), by coordinate conversion (transformation), pixel 
positions of the portions having distortion at the periphery of picture image, and cannot 
provide sufficient resolution and the portion having distortion is inferior as compared 
to the portion having no distortion from a viewpoint of picture quality. 

Disclosure of the Invention 

This invention has been made in view of actual circumstances as described 
above and its object is to provide a picture providing apparatus and a providing method 
therefor, and a picture processing apparatus and a precessing method therefor which 
can obtain picture image of high resolution without producing distortion. 

In addition, another object of this invention is to provide a picture providing 
apparatus and a providing method therefor, and a picture processing apparatus and a 
processing method therefor which are easy in maintenance, have high reliability, and 
can transmit picture image without producing time lag, wherein plural users can utilize 
images of view points or visual field angles of image pick-up different from each other. 
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This invention proposed in order to attain objects as described above is directed 
to a picture providing apparatus adapted to input distorted picture image in which a 
predetermined range is collectively imaged from image pick-up means to provide the 
entirety or a portion of the picture image in accordance with request of a picture 
display unit, the picture providing apparatus comprising: a transmit/receive unit 
adapted to receive, from the picture display unit, a request for transmission of picture 
image and to transmit, to the picture display unit, the entirety or a portion of picture 
image from which distortion is removed; a memory unit for storing picture image 
inputted from the image pick-up means; an arithmetic unit serving as selector means 
for selecting the entirety or a portion of picture image stored in the memory unit in 
correspondence with the request that the transmit/receive unit has received; and a 
picture conversion unit for eliminating distortion of the entirety or a portion of the 
picture image selected by the arithmetic unit and for converting it into high quality 
picture image. 

Moreover, this invention is directed to a picture image providing method for a 
picture image providing apparatus adapted for inputting distorted picture image in 
which a predetermined range is imaged from an image pick-up unit to provide the 
entirety or a portion of picture image in accordance with request of a picture image 
display unit, this method including: a communication step of receiving request for 
transmission of picture image from the picture image display unit and of transmitting 
the entirety or a portion of the picture image from which distortion has been 
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eliminated; a memory step of storing the picture image inputted from the image pick-up 
unit; a selection step of selecting the entirety or a portion of the picture image stored 
at the memory step in correspondence with the request received at the communication 
step; and a picture conversion step of converting the entirety or a portion of the picture 
image selected at the selection step into picture image from which distortion has been 
eliminated and of converting it into high quality picture image. 

Further, this invention is directed to a memory medium for providing that 
computer can read, the program being adapted to execute processing including: a 
communication step adapted to input distorted picture image in which a predetermined 
range is imaged from an image pick-up unit to receive, from a picture image display 
unit, request for transmission of picture image with respect to a picture providing unit 
for providing the entirety or a portion of picture image in accordance with request of 
the picture image display unit, and of transmitting the entirety or a portion of the 
picture image from which distortion has been eliminated to the picture image display 
unit; a memory step of storing picture image inputted from the image pick-up unit; a 
selection step of selecting the entirety or a portion of the picture image stored at the 
memory step; and a picture image conversion step of eliminating distortion of the 
entirety or a portion of the picture image selected at the selection step and of 
converting it into high quality picture image. 

Further, this invention is directed to a picture image processing apparatus, 
wherein this apparatus comprises an extraction circuit for extracting feature quantity 



every predetennined unit of picture image data having distortion, a classification circuit 
for carrying out classification every predetermined unit of picture data in accordance 
with the feature quantity extracted by the extraction circuit; and a picture image 
conversion unit for correcting distortion of picture data in accordance with result of the 
classification and for converting it into high quality picture image. 

Further, this invention is directed to a picture processing method, wherein this 
method includes an extraction step of extracting feature quantity every predetermined 
unit of picture data having distortion, a classification step of carrying out classification 
every predetennined unit of picture data in accordance with feature quantity extracted 
by this extraction step, and a picture image conversion step of correcting distortion of 
picture data in accordance with result of the classification and of converting it into high 
quality picture image. 

Further, this invention is directed to a memory medium, wherein there is stored 
program that computer can read, the program being adapted to execute processing 
including an extraction step of extracting feature quantity every predetennined unit of 
picture data having distortion, a classification step of carrying out classification every 
predetennined unit of picture data in accordance with the feature quantity extracted by 
the extraction step, and a picture image conversion step of correcting distortion of 
picture data in accordance with result of the classification and of converting it into high 
quality picture image. 

Further, this invention is directed to a picture image providing system composed 
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of an image pick-up unit, a picture providing unit and a picture display unit, wherein 
the image pick-up unit constituting this system includes image pick-up means for 
collectively imaging picture image of apredetenninedrange; the picture providing unit 
includes a first transmit/receive section adapted to receive request for transmission of 
picture image from the picture display unit and to transmit, to the picture display unit, 
the entirety or a portion of picture image from which distortion has been eliminated so 
that the picture image is caused to have high quality, a memory section for storing 
picture image inputted from the image pick-up unit, selector means for selecting the 
entirety or a portion of picture image that the memory section stores in correspondence 
with request that the first transmit/receive section has received, and a picture image 
converting circuit for eliminating distortion of the entirety or a portion of the picture 
image selected by the selector means and for converting it into high quality picture 
image; and the picture display unit includes a second transmit/receive section adapted 
to transmit request for transmission of picture image to the picture providing unit and 
to receive, from the picture providing unit, the entirety or a portion of picture image 
from which distortion has been eliminated so that the picture image is caused to have 
high quality. 

In addition, this invention is directed to a picture providing system comprising 
an image pick-up unit, a picture providing unit, and a picture processing unit, wherein 
the image pick-up unit constituting this system includes an image pick-up section for 
collectively imaging picture image of a predetermined range; the picture providing unit 
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includes a first transmit/receive section adapted to receive, from the picture image 
processing unit, request for transmission of picture image and to transmit the entirety 
or a portion of picture image to the picture processing unit, a memory section for 
storing picture image inputted from the image pick-up unit, and selector means for 
selecting the entirety or a portion of picture image that the memory section stores in 
correspondence with request that the first transmit/receive section has received; and 
the picture processing unit includes a second transmit/receive unit adapted to transmit, 
to the picture providing unit, request for transmission of picture image and to receive, 
from the picture providing unit, the entirety or a portion of the picture image, and a 
picture converting circuit for converting the received picture image into communication 
high quality picture image. 

Still more further objects of this invention and more practical merits obtained 
by this invention will become more apparent from the description of the embodiments 
which will be given below. 

Brief Description of the Drawings 

FIG. 1 is a block diagram showing a picture providing system according to this 
invention. 

FIG. 2 is a schematic perspective view showing an image pick-up unit according 
to this invention used in the picture providing system. 

FIG. 3 is a view showing picture image that the image pick-up um't outputs. 
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FIG. 4 is a block diagram showing a picture providing apparatus according to 
this invention used in the picture providing system. 

FIG. 5 is a view for explaining picture image which is read out from memory 

unit. 

FIG. 6 is a view for explaining picture image obtained by converting picture 
image which has been read out from the memory unit. 

FIG. 7 is a block diagram showing the configuration of picture processing unit 
according to this invention used in the picture providing system. 

FIG. 8 is a view for explaining block for classification and block for predictive 
value calculation. 

FIGS. 9 A and 9B are views for explaining processing of ADRC. 

FIGS. 10A and 10B are views for explaining classification processing. 

FIG. 1 1 is a block diagram showing learning unit according to this invention. 

FIG. 12 is a block diagram showing picture processing unit. 

FIG. 13 is a flowchart for explaining processing which provides picture image 
of the picture providing unit. 

FIG. 14 is a flowchart for explaining processing of conversion of picture image 
of the picture processing unit. 

FIG. 15 is a block diagram showing the configuration of computer into which 
program for executing a series of processing of the picture providing system is 
installed. 
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Best Mode for Carrying Out the Invention 

A picture providing apparatus and a providing method therefor, a picture 
processing apparatus and a processing method therefor, and a memory (recording) 
medium adapted so that program for carrying out picture processing is recorded 
according to this invention will now be described. 

First, the picture providing system to which this invention is applied will be 
described. This picture providing system comprises an image pick-up unit 1 as shown 
in FIG. 1 . The image pick-up unit 1 serves to pick up image of a predetennined obj ect 
to deliver the picked up moving picture to a picture providing apparatus (unit) 2. The 
picture providing unit 2 serves to store picture image delivered from the image pick-up 
unit 1 to extract a portion or the entirety of the stored picture image in correspondence 
with transmission request from a picture processing apparatus (unit) 4 or a monitor 7 
inputted through a network 3 to deliver it to the picture processing unit 4 or the monitor 
7 which has outputted transmission request through the network 3. 

The picture processing unit 4 inputs signal corresponding to request of user from 
an input unit 6- 1 to transmit instruction for making a request for transmission of picture 
image of a predetennined view point to the picture providing unit 2 through the 
network 3. Moreover, the picture processing unit 4 receives picture image delivered 
from the picture providing unit 2 through the network 3, i.e., picture image delivered 
in correspondence with instruction that the picture processing unit 4 has transmitted 
to convert that picture image or to output the picture image to a monitor 5 as it is. The 
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monitor 5 displays the picture image delivered from the picture processing unit 4. 

The monitor 7 inputs signal corresponding to request of user from an input unit 
6-2 to transmit instruction for making a request for transmission of picture image of a 
predetermined view point to the picture providing unit 2 through the network 3. In 
addition, the monitor 7 receives picture image delivered from the picture providing unit 
2 through the network 3, i.e., picture image delivered in correspondence with 
instruction that the monitor 7 has transmitted to display this received picture image. 

The image pick-up unit 1 used in this picture providing system comprises, as 
shown in FIG. 2, a transparent cylindrical body 1 formed by transparent glass or 
plastic, and a mirror 12 and a video camera 13 are supported within this cylindrical 
body 1 1 so that they are respectively located at predetermined positions. The mirror 
12 reflects light corresponding to image therearound from the external inputted through 
the transparent cylindrical body 11 to allow the video camera 13 to input it thereto. 
The video camera 13 picks up image therearound reflected by the mirror 12 to deliver, 
e.g., distorted picture image 2 1 shown in FIG. 3, for example, to the picture providing 
unit 2. Picture image that the video camera 1 3 picks up includes image of substantially 
360 degrees at the periphery of the optical axis of the video camera 13. 

The picture providing unit 2 constituting the picture providing system shown in 
FIG. 1 is constituted as shown in FIG. 4. This picture providing unit 2 comprises an 
A/D (Analog/Digital) converting section 31 supplied with picture signals from the 
image pick-up unit 1. The A/D converting section 31 carries out A/D conversion of 
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deformed picture image inputted from the image pick-up unit 1 to output digital data 
corresponding to inputted picture image to an arithmetic section 32. The arithmetic 
section 32 allows a memory section 33 to store digital data of picture image inputted 
from the AID converting section 3 1 . The memory section 33 stores digital data based 
on picture image delivered from the arithmetic section 32, e.g., the picture image 21 
shown in FIG. 3 to deliver, to the arithmetic section 32, digital data of picture image 
designated from the arithmetic section 32. 

A transmit/receive section 34 receives instruction for making a request for 
transmission of picture image of a predetermined view point from the picture 
processing unit 4 or the monitor 7 through the network 3 to deliver it to an address 
decoder 35. The address decoder 35 converts instruction delivered from the 
transmit/receive section 34 into code for designating address of picture image that the 
memory section 33 stores to output it to the arithmetic section 32. The arithmetic 
section 32 inputs code for designating address of picture image delivered from the 
address decoder 35 to read out picture image corresponding to that code from the 
memory section 33 to deliver it to a multiplexer (MPX.) 37 or a picture processing 
section 36. 

For example, as shown in FIG. 5, a portion or the entirety of picture image 21 
stored in the memory section 33 is specified by address of picture image delivered from 
the address decoder 35, and is read out from the memory section 33. For example, 
picture image 51 or picture image 52 is read out from the memory section 33, and is 
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delivered to the multiplexer 37 or the picture processing section 36. The picture 
processing section 36 converts picture image delivered from the arithmetic section 32 
into picture image having no distortion (distortion-free picture image) by classification 
adaptive processing to output it to the multiplexer 37. For example, picture image 5 1 
of FIG. 5 is converted into picture image 81 of FIG. 6, and picture image 52 of FIG. 
5 is converted into picture image 82 of FIG. 6. 

The multiplexer 37 selects, on the basis of control signal delivered from the 
arithmetic section 32, picture image delivered from the arithmetic section 32 or picture 
image delivered from the picture processing section 36 to output selected picture image 
to the transmit/receive section 34. The transmit/receive section 34 transmits picture 
image inputted from the multiplexer 37 to the picture processing unit 4 or the monitor 
7 which makes request for transmission of picture image. 

Explanation will now be given in more detail in connection with the picture 
processing section 36 adapted for converting picture image delivered from the 
arithmetic section 32 into picture image having no distortion (distortion-free picture 
image) by the classification adaptive processing. FIG. 7 is a block diagram showing 
the configuration of the picture processing section 36. A blocking circuit 101 for 
classification blocks distorted picture image delivered from the arithmetic unit 32 into 
blocks for classification in which remarked pixel which is unit for carrying out 
classification into predetermined classes in accordance with its property is being as 
center. 
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Namely, in FIG. 8, when it is assumed that pixel (portion indicated by black • 
mark in the Figure) which is located at i-th row from the top and j-th column from the 
left is represented as X ij5 the classification blocking circuit 101 constitutes 
classification blocks constituted by five pixels in total in which the own pixel Xg is 
added to four pixels X (i . 1)j5 X i(H> X i(j+1> X adjacent to the top, left, right and 
bottom portions thereof. This classification block is delivered to a classification 
adaptive processing circuit 103. 

In this case, the classification block is constituted by block consisting of pixels 
arranged in cross form. However, the shape of the classification block is not required 
to have cross shape. In addition thereto, e.g., rectangle, square or arbitrary shape may 
be employed. It is further noted that the number of pixels constituting classification 
block is not limited to five. 

A blocking circuit 102 for predictive value calculation blocks pixels of picture 
image into blocks for predictive value calculation in which remarked pixel, which is 
unit for calculating predictive value having no distortion, is caused to be center. 
Namely, in FIG. 8, when pixel values of 9 pixels of 3x3 in picture image having no 
distortion in which pixel X ;j (portion indicated by • mark in the figure) of picture 
image is caused to be center are respectively represented by Y Y »(2), Y s (3), Y 
ij(4), Y Sj (5), Y s (6), Y s (7), Y ^(8), Y y (9) in the right direction from the leftmost 
portion and in the lower direction from the uppermost portion, the predictive value 
calculation blocking circuit 102 constitutes square predictive value calculation blocks 
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constituted by, e.g., 22 pixels (X G . lxj _ 2) , X (i . 1)(j . 1)5 X m , X (i . m+1> X (i _ 1X j +2 ), X (i _i >cj + 3 > X 

(i-l)(j+4> X (j_i)(j +5 > X ( i.i )( j +6 ), X ( j.^Q+T), X i( j. 2) , X j( j. 1)5 Xjj, X i(j+1> X i(j+2)5 X ( i+ i)(j.2) 5 X 

1> X (Hl)p X (i+l)G+l> X (i+l)0+2> X (i+2)(j-l)> X (i+2)j> X (i+2)0+l) With the P ixel X ij bein S aS Cellter 

for the purpose of calculation of predictive values of pixels Yy(l) to Yjj(9). 

In more practical sense, e.g., for the purpose of calculation of predictive values 
of pixels Y 23 (l) to Y 23 (9) having no distortion encircled by square in FIG. 8, the 
predictive value calculation block is constituted by X n , X 12 , X 13 , X 14 , X 15 , X 16 , X 17 , X 18 , 

X 19' X 21> X 22> X 23» X 24' X 25j X 31> X 32' X 33s X 34j X 35> X 41> X 42j X 43- 

The predictive value calculation block obtained at the predictive value 
calculation blocking circuit 1 02 is delivered to classification adaptive processing circuit 
103. 

Also with respect to predictive value calculation block, the number of pixels and 
the shape are not limited to the above-described ones similarly to the case in the 
classification block. It is to be noted that it is desirable that the number of pixels 
constituting the predictive value calculation block is greater than the number of pixels 
constituting the classification block. 

In addition, there may change the number of pixels constituting the classification 
block and predictive value calculation block in dependency upon position of remarked 
picture image. 

There are instances where, while in the case where there is carried out blocking 
as described above (similarly also with respect to processing except for blocking), 
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corresponding pixel does not exist in the vicinity of picture frame of picture image. In 
this case, however, processing is carried out on the assumption that, e.g., the same 
pixels as pixels constituting picture frame exists at the outside thereof. 

The classification adaptive processing circuit 103 is composed of an ADRC 
(Adaptive Dynamic Range Coding) processing circuit 1 1 1, a classification circuit 1 12, 
a predictive coefficient memory circuit 1 13 and a predictive circuit 1 14, and serves to 
carry out classification adaptive processing. 

The ADRC processing circuit 111 implements ADRC processing to 
classification block from the classification blocking circuit 101 to deliver, to the 
classification circuit 1 12, block constituted with ADRC codes obtained as the result of 
ADRC processing. 

In this case, in accordance with the ADRC processing of the ADRC processing 
circuit 1 1 1, the number of bits of pixels constituting classification block is recorded 
from 8 bits to 2 bits. 

Namely, e.g., for the purpose of simplifying the explanation, as shown in FIG. 
9A, consideration is made with respect to block consisting of 4 pixels arranged on 
straight line. In the ADRC processing, maximum value MAX and lriinhnum value 
MIN of corresponding pixel value are detected. Further, DR==MAX- MIN is 
assumed to be local dynamic range of block. On the basis of the dynamic range DR, 
pixel value of pixels constituting block is requantized into K bits. 

Namely, the ininiiriuin value MIN is subtracted from respective pixel values 
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within block and the subtracted value i s divided by DR/2 K . Further, code (ADRC code) 
corresponding to divided value obtained as the result thereof is provided by conversion. 
In more practical sense, e.g., K is assumed to be equal to 2. In this case, as shown in 
FIG. 9B, it is judged that the divided value belongs to any one of ranges obtained by 
equally dividing the dynamic range DR by 4(=2 2 ). . In the case where the divided value 
belongs to the range of lowermost level, the range of the second level from the 
lowermost level, the range of the third level from the lowermost level and the range of 
the uppermost range, coding of 2 bits, e.g. 00B, 01B, 10B, 11B, etc. is provided (B 
indicates binary number). Further, on the decoding side, ADRC codes 00B, 0 IB, 10B, 
1 IB are converted into center value L 00 of the range of the lowermost level, center 
value L 01 of the second level from the lowermost level, center value L 10 of the range of 
the third level from the lowermost level, and center value L n of the range of the 
uppermost level which are obtained by equally dividing the dynamic range DR by 4. 
The minimum value MIN is added to these values so that decoding is carried out. 

Here, such an ADRC processing is called non-edge matching. 

It is to be noted that the detail of the ADRC processing is disclosed in, e.g., the 
Japanese Application Laid Open No. 53778/1991 publication that the applicant of this 
application has already filed, etc. 

In accordance with such an ADRC processing, requantization is carried out by 
the number of bits which is less than the number of bits assigned to pixels constituting 
block, thereby making it possible to reduce the number of bits. 
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Explanation will be briefly given in connection with the classification processing 
and the adaptive processing. The classification adaptive processing is a processing to 
classify input signal into several classes on the basis of the feature to implement 
adaptive processing suitable for those classes to input signal portions of respective 
classes, and is roughly classified into the classification processing and the adaptive 
processing. 

First, the classification processing will be described. 

Now, for example, as shown in FIG. 10A, block (block for classification) 
consisting of 2 x 2 pixels is assumed to be constituted by remarked pixel and three 
pixels adjacent thereto, and each pixel is assumed to be represented by 1 bit (takes any 
one of 0 or 1 level). In this case, block of 4 pixels of 2 x 2 can be classified into 16 ( 
(2 ! ) 4 ) patterns as shown in FIG. 1 OB by the level distribution of respective pixels. Such 
pattern classification is the classification processing and is carried out in the 
classification circuit 112. 

It is to be noted that the classification processing may be carried out in view of 
activity (complexity of picture image), (violence of change) of picture image (picture 
image within block), etc. 

In this example, ordinarily, e.g., about 8 bits are assigned to original pixel. 
Moreover, in this example, as described above, classification block consists of 5 pixels. 
Accordingly, when such classification block is constituted by original pixels of 8 bit to 
carry out classification processing, classification into vast quantity of (2 8 ) 5 is carried 
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out. 

On the other hand, in this example, in the ADRC processing circuit 1 1 1, the 
number of bits of pixels is reduced (reduced from 8 bits to 2 bits). Thus, the number 
of classes can be reduced (1024 (=(2 2 ) 5 ). 

The classification circuit 1 12 executes classification processing on the basis of 
ADRC code delivered from the ADRC processing circuit 1 1 1 to output a signal for 
designating class to predictive coefficient memory circuit 1 13. 

The adaptive processing will now be described. 

In order to carry out adaptive processing, it is necessary to make in advance 
learning of predictive coefficients for predicting picture image having no distortion 
from distorted picture image by distorted picture image inputted by the image pick-up 
unit 1 and distortion-free picture image taken in by plural cameras, etc. to store them 
in the predictive coefficient memory circuit 113. 

This adaptive processing will be explained by using a learning unit 200 shown 
in FIG. 11. 

From a distorted picture image input section 201, distorted picture image 
inputted by the image pick-up unit 1 is inputted. Moreover, from a distortion-free 
picture input section 202, distortion-free picture image at substantially the same 
position as the position which is imaged by the image pick-up unit 1 is inputted. For 
example, the distortion-free picture input section 202 is plural cameras having less 
periphery distortion. 
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Then, picture images inputted from the distorted picture image input section 20 1 
and distortion-free picture input section 202 are transferred to a blocking section 203 
for learning. The learning blocking section 203 generates picture blocks which are 
processing unit used for learning of predictive coefficients between distorted picture 
image and distortion-free picture image. In this case, the number of pixels of the 
distorted portion at the periphery of the distorted picture image is less as compared to 
corresponding number of pixels of distortion-free picture image. Thus, it is desirable 
to change picture image blocks for learning with respect to the portion in which picture 
image is not distorted and distorted portion of the distorted picture image. For 
example, it is desirable that there is set, with respect to the distorted portion, such a 
picture image block of broad range to include pixels of the portion away from the 
remarked pixel. In this connection, since prediction from picture block having lesser 
number of pixels to picture block having greater number of pixels is carried out at the 
stage of the adaptive processing, it is necessary to predict plural distortion-free pixel 
values by making use of the same distorted picture block. In view of the above, i.e., 
by respectively collecting picture blocks in which relative positions from remarked 
pixel are the same to learn predictive coefficients, it is possible to predict plural 
distortion-free pixel values from the same distorted picture block. 

Distorted picture block extracted at the learning blocking section 203 is inputted 
to an ADRC processing section 204 and is classified into classes every distorted picture 
block at a classification section 205 on the basis of ADRC processing result. The 
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ADRC processing section 204 and the classification section 205 are of the 
configurations similar to those of the above-described ADRC processing circuit 1 1 1 
and the classification circuit 1 12, respectively. 

Classification result of distorted picture block is inputted to a predictive 
coefficient learning section 206. Further, corresponding distorted picture block and 
distortion-free picture image serving as teacher of prediction are inputted to predictive 
coefficient learning section 206 through the learning blocking section. 

The predictive coefficient learning section 206 will now be described in detail. 

For example, here, let consider that predictive value E[y] of pixel value y of 
distortion-free picture image is detennined by linear first-order combination model 
prescribed by linear combination of pixel values of pixels of distorted picture image 
(hereinafter referred to as learning data as occasion may demand) x l5 x 2 , • • ■ .and 
predetermined predictive coefficients W l5 W 2 , • • • . In this case, the predictive value 
E[y] can be represented by the following equation. 

E[y] = WjX! + w 2 x 2 + • • • + w 25 x 25 ( 1 ) 

Now, matrix W consisting of set of predictive coefficients w, matrix X 
consisting of set of learning data and matrix Y' consisting of set of predictive values 
E[y] are defined as follows. 
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Observation equation as described below holds. 
XW = Y' 

(2) 

Thus, let consider that the least square is applied to this observation equation to 
detennine predictive value E[y] close to pixel value y of distortion-free picture image. 
In this case, matrix Y consisting of set of pixel values y of distortion-free picture image 
(hereinafter referred to as teacher data as occasion may demand), and matrix E 
consisting of set of residuals e of predictive value E[y] with respect to pixel value y of 
distortion-free picture image are defined as follows. 
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Residual equation as described below holds from the equation (2). 
XW = Y + E 

(3) 

In this case, predictive coefficient wi for determining predictive value E[y] close 
to pixel value y of distortion-free picture image can be detennined by rniniinizing sqare 
error as described below. 



Accordingly, in the case where value obtained by differentiating the above- 
described square error by predictive coefficients wi, i.e., the predictive coefficient wi 
which satisfies the following equation results in optimum value for detenriining 
predictive value E[y] close to pixel value y of distortion-free picture image. 



i ~ — + ei— — h 



-= 0(i= 1,2, ... ,n) 



(4) 

In view of the above, by differentiating the equation (3) by predictive 
coefficients wi, the following equation holds. 
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d~Wi ' d m>2 3 Wn 

(5) 

From the equations (4) and (5), the equation (6) is obtained. 

Yj ejcn = 0, J] &x>2 = 0, ... Y ejCi " - 0 

f=l ;=1 i=l 

(6) 

Further, when the relationship of learning data x, predictive coefficient w 5 teacher data 
y and residual e in the residual equation of the equation (3) is taken into consideration, 
normal equations as described below can be obtained. 

(Jj X,iX,i)w\ + (£ XnXi2)W2 + ... + (X x,ix m ) w„ = xnyd 
(£ x,2Xn)wi + (Y J x,2Xiz)W2 + xnXm)w n = (]£ xnyi) 

i=l »=1 j=1 ;=1 

(7) 



The normal equations of the equation (7) can be formed by the same number as 
the number of predictive coefficient w to be determined. 
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Then, the above -de scribed normal equations are transferred to a predictive 
coefficient arithmetic (computing) section 207. The predictive coefficient arithmetic 
section 207 can determine optimum predictive coefficient w by solving the equation 
(7). In solving the equation (7), e.g., sweep-out method (method of elimination of 
Gauss- Jordan), etc. can be applied. 

In a manner stated above, optimum coefficients w are determined every classes, 
and predictive coefficients are transferred to a predictive coefficient memory circuit 
113. 

Namely, the adaptive processing is processing in which predictive coefficients 
w obtained by learning stored in the predictive coefficient memory circuit 1 13 are used 
to determine predictive value E[y] close to pixel value y of distortion-free picture 
image from distorted picture image. 

The predictive coefficient memory circuit 113 shown in FIG. 7 stores in advance 
optimum predictive coefficients w every classes as described above. The predictive 
coefficient memory circuit 113 outputs, to predictive circuit 114, optimum predictive 
coefficients w every classes on the basis of signal for designating class delivered from 
classification circuit 1 12. The predictive circuit 1 14 executes adaptive processing on 
the basis of optimum predictive coefficient w delivered from the predictive coefficient 
memory circuit 113 to output pixel value y of distortion-free picture image to the 
multiplexer (MPX.) 37. 

It is to be noted that the adaptive processing differs from interpolation 
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processing in that there are reproduced components included in distortion-free picture 
image which are not included in distorted picture image. Namely, when only the 
equation (1) is viewed, the adaptive processing is similar to interpolation processing 
using so-called interpolation filter, but predictive coefficients w corresponding to tap 
coefficients of that interpolation filter are detennined by the so-called learning using 
teacher data y. For this reason, components included in distortion-free picture can be 
reproduced. From this fact, it can be said that the adaptive processing is processing 
having the so-called creative action of picture image. 

FIG. 12 is a block diagram for explaining the configuration of the function of 
the picture processing unit 4 when distorted picture image that the picture processing 
unit 4 receives from the picture providing unit 2 is converted into picture image having 
no distortion. A blocking circuit 131 for classification blocks pixels of distorted 
picture image received from the picture providing unit 2 through the network 3 into 
blocks for classification to deliver them to a classification adaptive processing circuit 
133. A blocking circuit 132 for predictive value calculation blocks pixels of distorted 
picture image received from the picture processing unit 2 through the network 3 into 
blocks for predictive value calculation to deliver them to the classification adaptive 
processing circuit 133. 

The classification adaptive processing circuit 133 is composed of an ADRC 
processing circuit 141, a classification circuit 142, a predictive coefficient memory 
circuit 143, and a predictive circuit 144, and serves to execute classification adaptive 
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processing on the basis of classification block delivered from the classification 
blocking circuit 131 and predictive value calculation block delivered from the 
predictive value calculation blocking circuit 132 to create distortion-free picture image 
to output it to the monitor 5. 

The ADRC processing circuit 141 implements ADRC processing to 
classification blocks from the classification blocking circuit 131 to deliver blocks 
constituted by ADRC codes obtained as the result thereof to the classification circuit 
142. The classification circuit 142 executes classification processing on the basis of 
ADRC codes delivered from the ADRC processing circuit 141 to output a signal for 
designating class to the predictive coefficient memory circuit 143. The predictive 
coefficient memory circuit 143 stores in advance optimum predictive coefficients w 
every classes to output, to the predictive circuit 144, optimum predictive coefficients 
w every classes on the basis of signal which designates class delivered from the 
classification circuit 142. The predictive circuit 144 executes adaptive processing on 
the basis of optimum predictive coefficient w delivered from the predictive coefficient 
memory circuit 143 and predictive value calculation block delivered from the 
predictive value calculation blocking circuit 132 to output pixel value y of distortion- 
free picture image to the monitor 5. 

The processing for providing picture image of the picture providing unit 2 will 
now be described with reference to the flowchart of FIG. 13. At step S 1 1 , manager of 
the picture providing unit 2 starts (activates) the picture providing unit 2. At step S 12, 
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an A/D conversion section 31 of the picture providing unit 2 carries out A/D 
conversion of picture image delivered from the image pick-up unit 1 to generate digital 
data corresponding to inputted picture image to deliver it to the arithmetic section 32. 
At step S13, the arithmetic section 32 of the picture providing unit 2 allows the 
memory section 33 to store digital data inputted from the A/D conversion section 31. 

At step S 14, transmit/receive section 34 receives transmission request forpicture 
image delivered from the picture processing unit 4 or the monitor 7 through the 
network 3 to deliver it to address decoder 35. The address decoder 35 converts 
transmission request for picture image delivered from the transmit/receive section 34 
into code which designates address of picture image to output it to the arithmetic 
section 32. The arithmetic section 32 judges, on the basis of signal delivered from the 
address decoder 35, whether or not request for transmission exists. In the case where 
it is judged that request for transmission exists, the processing proceeds to step S 1 5 to 
input code which designates address of picture image delivered from the address 
decoder 35. At step SI 6, the arithmetic section 32 reads out a predetennined picture 
image from the memory section 33 on the basis of code which designates address of 
picture image. 

At step SI 7, the arithmetic section 32 judges, on the basis of signal delivered 
from the address decoder 35, i.e., signal corresponding to transmission request of 
picture image delivered from the picture processing unit 4 or the monitor 7, whether 
or not request for picture image conversion exists. As a result, in the case where it is 
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judged that request for picture image conversion exists, the processing proceeds to step 
S18 to output, to the picture processing section 36, a predetermined picture image 
which has been read out from the memory section 33 to allow the picture processing 
section 36 to convert it into distortion-free picture image to allow the multiplexer 37 
to output the distortion-free picture image to the transmit/receive section 34. At the 
step S 1 7, in the case where it is judged that request for picture image conversion does 
not exist, conversion into distortion-free picture image is executed at the picture 
processing unit 4. Accordingly, the step S18 is skipped, and the arithmetic section 32 
outputs, to the multiplexer 37, a predetermined picture image which has been read out 
from the memory section 33 as it is. Thus, the multiplexer 37 is caused to output the 
distorted picture image to the transmit/receive section 34. 

At step SI 9, the transmit/receive section 34 transmits, to the picture processing 
unit 4 or the monitor 7 which has outputted transmission request for picture image is 
outputted at the step S 14, the picture image delivered from the multiplexer 37 through 
the network 3. At step S20, the arithmetic section 32 judges, on the basis of signal 
from input section (not shown), whether or not providing of picture image is 
completed. As a result, in the case where it is judged that providing of picture image 
is not completed, the processing returns to the step S12 to repeat processing of 
providing of picture image. At the step S20, in the case where it is judged that 
providing of picture image is completed, the processing is completed. 

At the step S14, in the case where it is judged that there is no request for 
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transmission, the procedure returns to the step SI 2. Thus, processing is repeated from 
input of picture image delivered from the image pick-up unit 1. 

As described above, the picture providing unit 2 provides a predetermined 
picture image with respect to the picture processing unit 4 or the monitor 7 on the basis 
of request from the picture processing unit 4 or the monitor 7. In the case where the 
picture processing unit 4 and the monitor 7 request different picture images at the same 
time, the picture providing unit 2 transmits predetermined pictures images at the same 
time. 

The processing of conversion of picture image of the picture processing section 
36 corresponding to the step SI 8 of FIG. 13 will now be described with reference to 
the flowchart of FIG. 14. At step S51, the picture processing unit 36 inputs a 
predetermined picture image from the arithmetic section 32. At step S52, the 
classification blocking circuit 101 blocks distorted picture image into classification 
blocks in which remarked pixel is caused to be center. The predictive value calculation 
blocking circuit 102 blocks distorted picture image into predictive value calculation 
blocks in which remarked pixel is caused to be center. 

At step S53, the ADRC processing circuit 1 1 1 generates ADRC code of 2 bits 
corresponding to pixel value of pixels of classification block by ADRC processing on 
the basis of classification block delivered from the classification blocking circuit 101. 
At step S54, the classification circuit 1 12 carries out classification on the basis of 
ADRC code delivered from the ADRC processing circuit 1 1 1 to output a signal for 
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designating class to the predictive coefficient memory circuit 113. 

At step S 5 5 , the predictive coefficient memory circuit 113 outputs, to predictive 
circuit 114, optimum predictive coefficients w every classes corresponding to signals 
which designate classes. At step S56, the predictive circuit 1 14 calculates pixel value 
of remarked pixel on the basis of optimum predictive coefficient w delivered from the 
predictive coefficient memory circuit 113 and predictive value calculation block 
delivered from the predictive value calculation blocking circuit 102. 

At step S57, the predictive circuit 1 14 outputs, to the multiplexer 37, pixel value 
of pixels of distortion-free picture image calculated at the step S56. At step S58, the 
predictive circuit 58 judges whether or not pixel value of pixels of a predetermined 
picture image is outputted. As a result, in the case where it is judged that pixel value 
of pixels of a predetermined picture image is not outputted, the processing procedure 
returns to the step S5 1 to repeat processing of conversion of picture image. At step 
S58, in the case where it is judged that pixel value of pixels of the predetennined 
picture image has been outputted, the processing procedure is completed. 

As stated above, the picture processing section 36 generates distortion-free 
picture image by the classification adaptive processing from picture image delivered 
from the arithmetic section 32. 

It should be noted that processing that the picture processing unit 4 carries out 
when the picture providing unit 2 transmits as it is, to the picture processing unit 4, a 
portion of picture image that the image pick-up unit 1 has imaged is similar to the 
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processing which has been explained with reference to the flowchart of FIG. 14. 

As stated above, the picture providing system of this invention can provide 
picture images at different positions or angles of visual field on the real time basis with 
respect to plural users. Accordingly, in the case where such system is utilized in the 
field of security, dead angle of camera is reduced and security is thus enhanced. In the 
case where such system is utilized in the field of advertisement, information guide or 
real estate, many users can view at the same time and at arbitrary position or angle of 
visual field of picture image such as actual goods or the inside of building, etc. 
Accordingly, quantity of information to be provided is increased. Thus, advertising 
effect is enhanced. 

Further, in the field of the remote medical service of which study is being put 
into practical use, in the case where plural physicians distributed at different places 
organize team to carry out operation, respective plural physicians can view, at the same 
time, picture images at the portions that they desire to confirm. Accordingly, 
respective physicians can carry out precise judgment in a short time. Thus, burden on 
patient is lessened. 

Since the image pick-up unit 1 has no mechanical operating portion, it is 
sufficient that interval of maintenance is long and the image pick-up unit 1 is less out 
of order. 

It is to be noted that while it is described that the image pick-up unit 1 picks up 
moving picture to output it to the picture providing unit 2, the image pick-up unit 1 may 
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pick up still picture to output it to the picture providing unit 2 and the picture providing 
unit 2 may provide still picture with respect to the picture processing unit 4 or the 
monitor 7. 

In addition, the image pick-up unit 1 may be also caused to be of configuration 
utilizing one fisheye lens or more. 

The system disclosed here represents the entirety of the apparatus constituted 
by plural units. 

Further, the above-described series of processing may be carried out by 
hardware, or may be carried out by software. In the case where a series of processing 
are carried out by software, program constituting that software is installed into widely 
used computer, etc. 

Thus, FIG. 1 5 shows an example of computer into which program for executing 
the above-described series of processing is installed. Program used here may be 
recorded in advance with respect to a hard disc 305 or a ROM 303 as recording 
medium included within the computer. Alternatively, program may be temporarily or 
permanently stored (recorded) with respect to removable recording medium such as 
floppy disc, CD-ROM (Compact Disc Read Only Memory), MO (Magneto Optical) 
disc, DVD (Digital Versatile Disc), magnetic disc, semiconductor memory, magnetic 
memory, or optical memory, etc. Such a removable recording medium can be provided 
as the so-called package software. 

It is to be noted that, in addition to installing program from removal recording 
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media as described above into computer, program may be transferred to computer by 
radio transmission (wireless system) through artificial satellite for digital satellite 
broadcast from down load site, or may be transferred to computer by wire through 
network such as internet, and computer may receive program transferred in such a 
manner by a communication section 308 to install such program onto the hard disc 305 
included therewithin. 

The computer includes therewithin a CPU (Central Processing Unit) 302. An 
input/output interface 310 is connected to the CPU 302 through a bus 301. When 
command is inputted to the CPU 302 as the result of the fact that an input section 307 
constituted with keyboard and/or mouse, etc. is operated by user through an 
input/output interface 3 10, the CPU 302 executes program stored in the ROM (Read 
Only Memory) 303 in accordance with such a command. Alternatively, the CPU 302 
loads, into a RAM (Random Access Memory) 304, program stored on the hard disc 
305, program transferred from satellite or network, received by the communication 
section 308 and installed on hard disc 305, or program installed on the hard disc 305 
which has been read out from removable recording medium attached to a drive 309 to 
execute such program. Thus, the CPU 302 carries out processing in accordance with 
the above-described flowchart, or processing carried out by the configuration of the 
above-described block diagram. Further, e.g., the CPU 302 outputs its processing 
result from an output section 306 comprised of LCD (Liquid Crystal Display) or 
speaker, etc. through the input/output interface 310, transmits it from the 
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communication section 308, and/or record it onto the hard disc 305, etc. as occasion 
demand. 

It is here noted that while it is not necessarily required to process processing 
steps for describing program for allowing computer to carry out various processing in 
a time series manner along the sequence described as flowchart, such processing may 
include processing executed in parallel or individually, e.g., parallel processing or 
processing by object. 

Further, program may be processed by one computer, or may be processed in 
a distributed manner by plural computers. In addition, program may be transferred to 
computer at remote place, at which it is executed. 

Industrial Applicability 

In this invention, since learning is carried out between distorted picture image 
and distortion-free high quality picture, distorted picture image is corrected so that 
there results distortion-free picture image and is collectively converted into high quality 
picture image. Thus, resolution of distorted picture image can be converted into 
resolution similar to the portion free from distortion. Since conversion can be made 
by only predictive operation without a processing to carry out interpolation after 
coordinate transformation (conversion), it does not take much time in operation. In 
addition, unlike interpolation conventionally used, predictive operation is carried out 
on the basis of learning between distorted picture image and distortion-free picture 
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image. Accordingly, this system is equivalent to creation of pixels which do not exist. 
Thus, picture image of higher resolution can be obtained. 
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CLAIMS 

1 . A picture providing apparatus adapted to input distorted picture image in which 
a predetermined range is collectively imaged from image pick-up means to provide the 
entirety or a portion of the picture image in accordance with request of a picture image 
display unit, 

the apparatus including: 

communication means for receiving request for transmission of picture image 
from the picture image display unit and for transmitting the entirety or a portion of the 
picture image from which distortion has been eliminated to the picture image display 
unit; 

memory means for storing the picture image inputted from the image pick-up 
means; 

selector means for selecting the entirety or a portion of the picture image stored 
in the memory means in correspondence with the request that the communication 
means has received; and 

picture image conversion means for eliminating distortion of the entirety or a 
portion of the picture image selected by the selector means and for converting it into 
high quality picture image. 

2. The picture providing apparatus as set forth in claim 1, 

wherein the picture image conversion means eliminates distortion of the entirety 
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or a portion of the picture image selected by the selector means by classification 
adaptive processing and converts such picture image into high quality picture image. 



3. The picture providing apparatus as set forth in claim 2, 

wherein the picture image conversion means reduces the number of bits of pixel 
value of pixels of block for classification by ADRC processing. 

4. A picture providing method for a picture providing apparatus adapted to input 
distorted picture image in which a predetermined range is collectively imaged from an 
image pick-up unit to provide the entirety or a portion of the picture image in 
accordance with request of a picture display unit, 

the method including: 

a communication step of receiving request for transmission of picture image 
from the picture image display unit and of transmitting the entirety or a portion of the 
picture image from which distortion is elhninated; 

a memory step of storing the picture image inputted from the image pick-up unit; 

a selection step of selecting the entirety or a portion of the picture image stored 
at the memory step in correspondence with the request received at the communication 
step; and 

a picture image conversion step of converting the entirety or a portion of the 
picture image selected at the selection step into picture image from which distortion 
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has been eliminated and of converting it into high quality picture image. 

5 . A memory medium adapted so that there is provided program that computer can 
read, 

processing executed by the computer including: 

a communication step of inputting distorted picture image in which a 
predetermined range is collectively imaged from an image pick-up unit to receive 
request for transmission of picture image from a picture image display unit to a picture 
providing unit for providing the entirety or a portion of the picture image in accordance 
with request of the picture image display unit, and of transmitting the entirety or a 
portion of the picture image from which distortion is removed to the picture image 
display unit; 

a memory step of storing the picture image inputted from the image pick-up unit; 

a selection step of selecting the entirety or a portion of the picture image stored 
at the memory step in correspondence with the request received at the communication 
step; and 

a picture image conversion step of eliminating distortion of the entirety or a 
portion of the picture image selected at the selection step and of converting it into high 
quality picture image. 



6. 



A picture processing apparatus comprising: 
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extraction means for extracting feature quantity every predetermined unit of 
picture data having distortion; 

classification means for carrying out classification every predetemiined unit of 
the picture data in accordance with the feature quantity extracted by the extraction 
means; and 

picture image conversion means for correcting the distortion of the picture data 
in accordance with result of the classification and for converting the picture data into 
high quality picture image. 

7. The picture processing apparatus as set forth in claim 6, 

which further comprises image pick-up means for imaging the picture data 
having distortion. 

8. The picture processing apparatus as set forth in claim 6, 

wherein the picture image conversion means eliminates distortion of the picture 
image having distortion by classification adaptive processing and converts it into highy 
quality picture image. 

9. The picture processing apparatus as set forth in claim 6, 

wherein the classification means reduces the number of bits of pixel value of 
pixels of block for classification by the ADRC processing. 
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10. A picture processing method including: 

an extraction step of extracting feature quantity every predetermined unit of 
picture data having distortion; 

a classification step of carrying out classification every predetennined unit of 
the picture data in accordance with feature quantity extracted by the extraction step; 
and 

a picture image conversion step of correcting the distortion of the picture data 
in accordance with result of the classification and of converting it into high quality 
picture image. 

11. A memory medium adapted so that there is provided program that computer can 
read, 

processing executed by the computer including; 

an extraction step of extracting feature quantity every predetennined unit of 
picture data having distortion; 

a classification step of carrying out classification every predetermined unit of 
the picture data in accordance with the feature quantity extracted by the extraction 
step; and 

a picture image conversion step of correcting the distortion of the picture data 
in accordance with result of the classification and of converting it into high quality 
picture image. 
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12. A picture providing system comprising an image pick-up unit, a picture 
providing unit, and a picture display unit, 

wherein the image pick-up unit includes image pick-up means for collectively 
imaging picture image of a predetermined range, 

wherein the picture providing unit includes: 

first communication means for receiving request for transmission of picture 
image from the picture display unit and for transmitting, to the picture display unit, the 
entirety or a portion of the picture image from which distortion has been eliminated so 
that there is provided high quality picture image; 

memory means for storing the picture image inputted from the image pick-up 

unit; 

selector means for selecting the entirety or a portion of the picture image that 
the memory means stores in correspondence with the request that the first 
communication means has received; and 

picture image conversion means for eliminating distortion of the entirety or a 
portion of the picture image selected by the selector means and of converting it into 
high quality picture image, and 

wherein the picture display unit includes 

second communication means for transmitting request for transmission of the 
picture image to the picture providing unit and for receiving, from the picture providing 
unit, the entirety or a portion of the picture image from which distortion is eluninated 
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so that there is provided high quality picture image. 

13. A picture providing system comprising an image pick-up unit, a picture 
providing unit and a picture processing unit, 
wherein the image pick-up unit includes 

image pick-up means for collectively imaging picture image of a predetennined 

range, 

wherein the picture providing unit includes: 

first communication means for receiving request for transmission of picture 
image from the picture processing unit and for transmitting the entirety or a portion of 
the picture image to the picture processing unit; 

memory means for storing the picture image inputted from the image pick-up 
unit; and 

selector means for selecting the entirety or a portion of the picture image that 
the memory means stores in correspondence with the request that the first 
communication means has received, and 

wherein the picture processing unit includes: 

second communication means for transmitting request for transmission of the 
picture image to the picture providing unit and for receiving the entirety or a portion 
of the picture image from the providing unit; and 

picture image conversion means for carrying out conversion into the high quality 



picture image. 
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ABSTRACT 

In this invention, a transmit/receive section 34 receives, from monitor, request 
for transmission of picture image through network and transmits, to the monitor, the 
entirety or a portion of picture image from which distortion has been eliminated. A 
memory section 33 stores picture image inputted from an image pick-up unit. An 
arithmetic section 32 selects the entirety or a portion of picture image stored in the 
memory section 33 in correspondence with request that the transmit/receive section 34 
has received. A picture processing section 36 converts the entirety or a portion of 
picture image selected by the arithmetic section 32 into picture image from which 
distortion has been eliminated. 
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Under the Paperwork Reduction Act of 1995, n< 



PTO/SB 106 (5-00) 
Approved for use through 10/31/02 OMB 0651-0032 
Patent and Trademark Office: U.S. DEPARTMENT OF COMMERCE 
re required to respond to a collection of information unless it displays a valid OMB control number 



Declaration and Power of Attorney for Patent Application 

Japanese Language Declaration 



s inventor, I hereby declare that: 



My residence, post office address and citizenship are as stated 
next to my name: 



«T*8 («-«JS**f|SR*hTV»SJ|^) )S\ «V>l*ft&K a* 



I believe I am the original, first and sole inventor(if only one 
name is listed below) or an original, first and joint inventor (if 
plural names are listed below) of the subject matter which is 
claimed and for which a patent is sought on the invention 
entitled. 

PICTURE PROVIDING APPARATUS AND PROVIDING 
METHOD THEREFOR, PICTURE PROCESSING 

APPARATUS AND PROCESSING METHOD THEREFOR, 
AND MEMORY MEDIUM 



ffl B « # m ffl K # t * n ft P C T EE! IS ffi K * * 1 i . 



hjw«s«wu flirts tan it w* c h t c ctcasst « . 



1te-n'T«B«tflllL*MS***S*c**Ci:*B**. 



the specification of which is attached hereto unless the 
following box is checked: 



X was filed on March 13. 2000 

as United States Application Number of 

PCT International Application Number PCT/JP0O/O151 1 

and was amended on 

(if applicable). 



I hereby state that I have reviewed and understand the contents 
of the above identified specification, including the claims, as 
amended by any amendment referred to above. 



I acknowledge the duty to disclose information which is 
material to patentability as defined in Title 37, Code of Federal 
Regulations, Section 1.56. 



Burden Hour Statement: This form is estimated to take 0 4 hours to complete. Time will vary depending upon the need of the individual case. Any comments on the 
amount of time you are required to complete this form should be sent to Chief Information Officer. U.S. Patent and Trademark Office, Washington. DC 2023 1 . DO 
NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner of Patents and Trademarks, Washington, DC 20231 . 
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Japanese Language Declaration 



SJK. *»tt*BIAJ^<»^ft< 4 4-Bf*ffl3£l,TH$*BI£*J83 
6 5&<a)<ciSP CTEIISffiOtC^ViT. HIS 1 1 9« (a) 

Prior Foreign Application(s) 



J apan 
(Country) 
OS) 
PCT 

(Country) 



P CT/JTOO/ 01511 
(Number) 



I hereby claim foreign priority under Title 35, United States Code, 
"Section 1 19(a)-(d) or 365(b) of any foreign application(s) for patent or 
inventor's certificate, or 365(a) of any PCT International application 
which designated at least one country other than the United States listed 
below and have also identified below, by checking the box, any foreign 
application for^ patent or inventor's certificate or PCT international 
application having a filing date before that of the application for which 
priority is claimed. 

Priority Not Claimed 



12 March 1999 
(Day/Month/Year Filed) 



13 March 2000 



(Day/Month/Year Filed) 



(Number) 
. <**) 



(JMumber) 



(Number) 



(Country) 



(Country) 
(SI*) 



(Country) 
(12*) 



(Day/Month/Year Filed) 



(Day/Month/Year Filed) 



(Day/Month/Year Filed) 



□ 
□ 
□ 



(Kpmber) 
ffig-9) 



(Country) 
(H*) 



(Application No.) 



(Filing Date) 
(ffiOS) 



™ ? i£tt. ucc, TiB<ov>*^ J ;5j(tr3)ai|((; 

5jem i 2 o fiia*±iSL v 

3 6KS1 1 2£SH8fcJ!HS§ft*:fi«X\ 
fflH H fc * iHH B * fc (i P C T BtlStHK H 



■PV»T1>, + <93)S1S2£ 

s ss (c)K»-r?< ma 



(Day/Month/Year Filed) 



I hereby claim the benefit under Title 35. United States Code, Section 
1 19(e) of any united States provisional application(s) listed below. 



(Application No.) 



(Filing Date) 
(ffiHH) 



I hereby claim the benefit under Title 35, United States Code, Section 
120 of any United States appltcation(s), or 365© of any PCT 
international application designating the United States, listed below and, 
insofar as the subject matter of each of the claims of this application is 
not disclosed in the prior United States or PCT International application 
in the manner provided by the first paragraph of Title 35, United States 
Section 1 12, I acknowledge the duty to disclose information which is 
material to patentability as defined in Title 37, Code of Federal 
Regulations, Section 1.56 which became available between the filing 
date of the prior application and the national or PCT international filing 
date of application. 



(Application No.) 



(Filing Date) 
(fflHB) 



(Status: Patented, Pending, Abandoned) 



aoffiKt fa-fa c 1 Ha*, s hs t; c t 

fctt-thK»tT»fr SHiUfrtEafcHnk *03»ttteHB#£ 



I hereby declare that all statements made herein of my own 
knowledge are true and that all statements made on information 
and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or 
imprisonment, or both, under Section 1001 of Title 18 of the 
United States Code and that such willful false statements may 
jeopardize the validity of the application or any patent issued 
thereon. 
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Japanese Language Declaration 



0£x<o*B*mi-it:inz> a hums tlx. te<o# 



^7W 



POWER OF ATTORNEY: As a named inventor, I hereby appoint 
the following attorney(s) and/or agent(s) to prosecute this application 
and transact al business in the Patent and Trademark Office 
connected therewith (list name and registration number) 



WILfelAM S. FROMMER, Registration N o. 25,506 and 
>ENNIS M. SMID, Registration No. 34,93 ft 



Send Correspondence to: 
WILLIAM S. FROMMER, Esq. 
c/o FROMMER T.AWRF.NPF. ». HAJJIULL E- 
_ 745 Fifth Avenue , 



New York, New York 10151 



Direct Telephone Calls to: 
(212) 588-0800 
to the attention of: 
WILLIAM S. FROMMER 



Full name of sole or first inventor 



Tetsuiiro KONDO 



inventor's signature 



Residence u 



^Tpkyo^Japan 



Citizenship 
Japan 



Date 



Post Office Address 

c/o Sony Corporation, 7-35, KitasrrLnagawa 6-chcme, 
Shinagawa-ku, Tokyo 141-0001 Japan 



IS ~~~ 



urn 



full name of second joint inventor, if any 
JUnichi ISHIBASHI 



OW Second Inventor's signature Date 



j-iaitarra, Japan 
Citizenship 

Japan 



A3L 



Post Office Address 

c/o Sony Corporation, 7-35, Kitashinagawa 6-chome, 
Shinagawa-4ai, Tokyo 141-0001 Japan 



(Supply similar information and signature for third and subsequent 
joint inventors) 
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